feat(ai-gateway): add organization auto model routing by pandemicsyn · Pull Request #4056 · Kilo-Org/cloud

pandemicsyn · 2026-06-17T04:49:42Z

Summary

Adds kilo-auto/org, an Enterprise-only Organization Auto model that lets an organization choose which model should handle each Kilo mode.
kilo-auto/org is intentionally used rather than an org-name-derived ID: organization names are org owner mutable and not unique, and identity comes from authenticated organization context. This avoids rename migrations, stale cache keys, privacy leakage, and unbounded telemetry/catalog cardinality. Also, I think this name fits because its kilo’s “org” auto routing model.
Organization Auto stores routes in organizations.settings.org_auto_model:
- If a request mode has an explicit route, Organization Auto uses that model.
- If a request mode has no explicit route, Organization Auto uses the configured fallback model.
- If a request uses an unknown/custom mode, Organization Auto also uses the fallback model unless that exact mode slug has a route.
- Exact route slugs win first, so build uses routes.build before falling back to routes.code, and plan uses routes.plan before falling back to routes.architect.
Replaces the unshipped org-default-model-config mode-default rollout with the new organization-auto-model-routing PostHog toggle for canonical organization-wide routing.
Adds Enterprise-only enable/disable/fallback controls, canonical mode route configuration, route-aware mode lifecycle behavior, and exact route audit messages.
Replaces the default-model dialog with one unified behavior save flow for Organization Auto, a specific model, or the global default.
Organization Auto is presented as the recommended path with accessible radio semantics, keyboard navigation, and explicit selected-state affordances.
Reset now explicitly disables Organization Auto and returns the organization to global-default resolution while preserving dormant routes and fallback settings for future re-enable.
Custom Modes now labels stored routes as inactive when Organization Auto is off instead of implying they are currently applied.
Removes the unshipped OrganizationModeConfig.defaultModel bridge and stores all routing only in organizations.settings.org_auto_model.
Adds organization-owned direct BYOK route support with credential ownership checks, direct-provider routing, zero Kilo hosted cost, and no fallback to OpenRouter/Vercel when credentials disappear.
Adds client-safe shared Organization Auto helpers so browser components do not import server-only DB/pg modules.

Verification

Verified the default behavior chooser in the browser, including selected-state styling, the Recommended badge, and focused-but-unselected options.
Verified keyboard navigation for the radio chooser with ArrowUp/Down/Left/Right, Home, and End.
Verified Organization Auto and Specific model dialog states at the compact dialog width.
Verified Organization Auto API behavior manually with /tmp/org-auto-api-log.sh against a local service, including modes/defaults/models and optional gateway requests.

Visual Changes

Loom:

https://www.loom.com/share/30ded7b1543c4b2aa0d1e5989dc04d68

Reviewer Notes

Organization Auto UI/configuration writes are gated by the PostHog release toggle organization-auto-model-routing, replacing the old org-default-model-config toggle. Runtime routing remains data-driven for already-configured organizations, and disable remains available during rollback.
Stripe webhook/admin plan transitions intentionally do not clear the stored sentinel; catalog/default/runtime eligibility gates make dormant routes safe while preserving future re-enable behavior. The explicit global reset path now disables active Organization Auto while preserving those dormant route settings.
Review BYOK fallback protection, and route-management authorization as the highest-risk areas.

kilo-code-bot · 2026-06-17T04:58:31Z

Code Review Summary

Status: 1 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
`apps/web/src/routers/cli-sessions-router.ts`	290	Mixed `other` + explicit platform filters still collapse to `IN (...)`, so selections like `['other','github']` omit unknown-platform sessions instead of OR-ing `NOT IN KNOWN_PLATFORMS` with the concrete values.

No current gh pr diff lines were commentable for this carried-forward issue. The incremental changes in 67a608ccb897b9301e9ae8c193f2a19afc3c5662 did not introduce any new findings.

Files Reviewed (2 files)

apps/web/src/app/api/openrouter/[...path]/route.ts
apps/web/src/lib/ai-gateway/auto-model/index.ts

Previous Review Summaries (7 snapshots, latest commit 9f9f610)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit `9f9f610`)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

No current gh pr diff lines were commentable for this issue.

Other Observations (not in diff)

Issues found in changed code that no longer appears in the current PR diff and therefore could not receive an inline comment:

File	Line	Issue
`apps/web/src/routers/cli-sessions-router.ts`	290	Mixed `other` + concrete platform filters still use `IN (...)`, so selections like `['other','github']` drop unknown-platform sessions instead of OR-ing `NOT IN KNOWN_PLATFORMS` with the concrete values

Files Reviewed (115 files)

apps/web/src/routers/cli-sessions-router.ts - 1 warning
114 additional incremental files reviewed with no new findings

Previous review (commit `f753b11`)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Single commit addresses the previously flagged audit log issue by capturing previousDefaultModel inside the mutation callback before the organization settings are modified. A new test assertion verifies the correct audit message for the Organization Auto → global transition. No new issues found.

Files Reviewed (2 incremental, 42 total)

apps/web/src/routers/organizations/organization-settings-router.ts - Fixed audit log (previous concern resolved)
apps/web/src/routers/organizations/organization-settings-router.test.ts - Added audit log assertion

Previous review (commit `cf3f457`)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Refactors default model mutations to use mutateOrganizationSettings with for update locking, adds Organization Auto routing with BYOK support, and introduces a unified configureOrganizationDefaultBehavior endpoint. Previous audit log concern at line 878 resolved by the refactor — the pre-transaction existingOrg is now used correctly to determine audit message type. No new issues found.

Files Reviewed (42 files)

apps/web/src/app/(app)/organizations/[id]/custom-modes/page.tsx
apps/web/src/app/api/openrouter/[...path]/route.test.ts
apps/web/src/app/api/openrouter/[...path]/route.ts
apps/web/src/app/api/organizations/[id]/defaults/route.test.ts
apps/web/src/app/api/organizations/[id]/defaults/route.ts
apps/web/src/app/api/organizations/[id]/modes/route.test.ts
apps/web/src/app/api/organizations/[id]/modes/route.ts
apps/web/src/app/api/organizations/hooks.ts
apps/web/src/components/models/CondensedProviderAndModelsList.tsx
apps/web/src/components/organizations/OrganizationDashboard.tsx
apps/web/src/components/organizations/OrganizationProvidersAndModelsConfigurationCard.tsx
apps/web/src/components/organizations/custom-modes/CustomModesLayout.tsx
apps/web/src/components/organizations/custom-modes/EditModeForm.test.ts
apps/web/src/components/organizations/custom-modes/EditModeForm.tsx
apps/web/src/components/organizations/custom-modes/ModeForm.tsx
apps/web/src/components/organizations/custom-modes/NewModeForm.tsx
apps/web/src/components/organizations/providers-and-models/DefaultModelDialog.tsx
apps/web/src/components/organizations/providers-and-models/ModelsTab.tsx
apps/web/src/components/organizations/providers-and-models/OrganizationProvidersAndModelsPage.tsx
apps/web/src/components/organizations/providers-and-models/ProvidersTab.tsx
apps/web/src/lib/ai-gateway/auto-model/index.ts
apps/web/src/lib/ai-gateway/auto-model/resolution.test.ts
apps/web/src/lib/ai-gateway/auto-model/resolution.ts
apps/web/src/lib/ai-gateway/byok/index.ts
apps/web/src/lib/ai-gateway/llm-proxy-helpers.ts
apps/web/src/lib/ai-gateway/providers/openrouter/index.ts
apps/web/src/lib/organizations/organization-auto-model-shared.ts
apps/web/src/lib/organizations/organization-auto-model.ts
apps/web/src/lib/organizations/organization-base-types.ts
apps/web/src/lib/organizations/organization-models.ts
apps/web/src/lib/organizations/organization-modes.test.ts
apps/web/src/lib/organizations/organization-modes.ts
apps/web/src/lib/organizations/organization-seats.ts
apps/web/src/lib/organizations/organization-types.ts
apps/web/src/lib/organizations/organizations.ts
apps/web/src/lib/proxy-error-types.ts
apps/web/src/routers/organizations/organization-modes-router.test.ts
apps/web/src/routers/organizations/organization-modes-router.ts
apps/web/src/routers/organizations/organization-settings-router.test.ts
apps/web/src/routers/organizations/organization-settings-router.ts
packages/db/src/schema-types.test.ts
packages/db/src/schema-types.ts

Previous review (commit `29a354e`)

Status: 1 Issue Found | Recommendation: Approve

Executive Summary

Refactors the default model dialog and custom modes UI with a unified behavior endpoint (auto/specific/global). One minor suggestion: the audit log inside the transaction uses pre-transaction state for the global-reset case.

Overview

Severity	Count
CRITICAL	0
WARNING	0
SUGGESTION	1

Issue Details (click to expand)

SUGGESTION

File	Line	Issue
`apps/web/src/routers/organizations/organization-settings-router.ts`	878	Audit log uses pre-transaction `existingOrg` instead of fresh `organization` parameter

Files Reviewed (9 files)

apps/web/src/app/api/organizations/hooks.ts
apps/web/src/components/models/CondensedProviderAndModelsList.tsx
apps/web/src/components/organizations/custom-modes/CustomModesLayout.tsx
apps/web/src/components/organizations/custom-modes/EditModeForm.tsx
apps/web/src/components/organizations/custom-modes/ModeForm.tsx
apps/web/src/components/organizations/custom-modes/NewModeForm.tsx
apps/web/src/components/organizations/providers-and-models/DefaultModelDialog.tsx
apps/web/src/routers/organizations/organization-settings-router.test.ts
apps/web/src/routers/organizations/organization-settings-router.ts - 1 issue

Fix these issues in Kilo Cloud

Previous review (commit `412123c`)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental changes cover a major tier-to-taxonomy refactoring across auto-routing contracts, decision engine, benchmarks, and admin UI, plus KiloClaw billing recovery UX, model catalog refresh for vision capabilities, cloud agent fork review flow, and CI/CD staging deployment. All changes are well-tested with no security, logic, or performance issues detected.

Files Reviewed (40+ files changed since last review)

packages/auto-routing-contracts/src/ — Tier → taxonomy refactoring (tiers.ts → taxonomy.ts + reasoning.ts)
services/auto-routing/src/ — Decision engine, tests, and index updated for taxonomy routes
services/auto-routing-benchmark/src/ — Decider sharding, org context, chunk pipeline rework
services/auto-routing-benchmark/**/migrations/ — D1 column renames (tier → route_key), benchmark_org_id
services/auto-routing-benchmark/container/ — kiloApiUrl + orgId support in CLI container
services/auto-routing-benchmark/wrangler.jsonc — max_instances 100, max_concurrency 100, retry tuning
services/kiloclaw/controller/src/model-catalog-refresh.ts — New periodic model catalog seed for vision
services/kiloclaw/controller/src/index.ts — Wiring for catalog refresh
apps/web/src/lib/ai-gateway/ — Efficient model now public, data collection check refactored, mirror removed
apps/web/src/lib/ai-gateway/auto-routing-decision.ts — buildDecidePayload inlined from deleted mirror module
apps/web/src/lib/ai-gateway/is-free-model.ts — hasBestEffortGuessDataCollectionRequirement
apps/web/src/lib/ai-gateway/providers/ — MiniMax m3 reasoning fix, GLM model variant fix, efficient model listing
apps/web/src/app/admin/auto-routing/ — Admin UI for taxonomy routes, benchmarkOrgId, costPerAccuracy
apps/web/src/app/api/internal/auto-routing-benchmark/token/ — Org-scoped token minting
apps/web/src/app/cloud-agent-fork/review/ — New fix-review redirect flow with prompt-based sessions
apps/web/src/lib/code-reviews/prompts/fix-review-prompt.ts — New prompt builder for review fix workflow
apps/web/src/lib/code-reviews/review-memory/github-feedback.ts — Repository permission gating for feedback recording
apps/web/src/lib/integrations/platforms/github/adapter.ts — Exported CollaboratorPermission type
apps/web/src/lib/kiloclaw/credit-billing.ts — CreditEnrollmentError with typed reasons, instance/price version gating
apps/web/src/lib/kiloclaw/pricing-catalog.test.ts — resolveKiloClawEnrollmentPriceVersion tests
apps/web/src/lib/kiloclaw/stripe-handlers.ts — Conditional logging on subscription.created processing
apps/web/src/app/(app)/claw/components/billing/ — Funding choice copy, hosting recovery UX
apps/web/src/app/payments/kilo-pass/awarding/ — Recovery UI with per-reason copy and retry
apps/web/src/routers/kilo-pass-router.ts — Structured activateCheckoutHosting with retryable/action_required outcomes
apps/web/src/routers/kiloclaw-router.ts — Enrollment error classification, destroyed/canceled gating
apps/web/src/routers/kiloclaw-billing-router.test.ts — Tests for checkout rejections and webhook ordering
packages/db/src/kiloclaw-pricing-catalog.ts — resolveKiloClawEnrollmentPriceVersion helper
.github/workflows/ — Staging deployment workflow, target_environment parameterization
apps/web/src/app/(app)/claw/components/changelog-data.ts — Vision fix changelog entry

Previous review (commit `b3107a9`)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The previous WARNING about getBYOKforOrganization reading outside transaction isolation has been fixed. validateOrganizationAutoTarget now accepts an optional dbClient and all transaction-bound callers correctly pass tx. No new issues detected in the incremental changes.

Files Reviewed (4 files changed since last review)

apps/web/src/lib/ai-gateway/byok/index.ts
apps/web/src/lib/organizations/organization-auto-model.ts
apps/web/src/routers/organizations/organization-modes-router.ts
apps/web/src/routers/organizations/organization-settings-router.ts

Previous review (commit `6e0261a`)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

validateOrganizationAutoTarget reads BYOK credentials outside the mutateOrganizationSettings transaction isolation context, creating a narrow consistency gap during Organization Auto route configuration.

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
`apps/web/src/lib/organizations/organization-auto-model.ts`	85	`getBYOKforOrganization(db, ...)` uses global `db` inside transaction callbacks, bypassing isolation

Files Reviewed (41 files)

apps/web/src/app/(app)/organizations/[id]/custom-modes/page.tsx
apps/web/src/app/api/openrouter/[...path]/route.test.ts
apps/web/src/app/api/openrouter/[...path]/route.ts
apps/web/src/app/api/organizations/[id]/defaults/route.test.ts
apps/web/src/app/api/organizations/[id]/defaults/route.ts
apps/web/src/app/api/organizations/[id]/modes/route.test.ts
apps/web/src/app/api/organizations/[id]/modes/route.ts
apps/web/src/app/api/organizations/hooks.ts
apps/web/src/components/models/CondensedProviderAndModelsList.tsx
apps/web/src/components/organizations/OrganizationDashboard.tsx
apps/web/src/components/organizations/OrganizationProvidersAndModelsConfigurationCard.tsx
apps/web/src/components/organizations/custom-modes/CustomModesLayout.tsx
apps/web/src/components/organizations/custom-modes/EditModeForm.test.ts
apps/web/src/components/organizations/custom-modes/EditModeForm.tsx
apps/web/src/components/organizations/custom-modes/ModeForm.tsx
apps/web/src/components/organizations/custom-modes/NewModeForm.tsx
apps/web/src/components/organizations/providers-and-models/DefaultModelDialog.tsx
apps/web/src/components/organizations/providers-and-models/ModelsTab.tsx
apps/web/src/components/organizations/providers-and-models/OrganizationProvidersAndModelsPage.tsx
apps/web/src/components/organizations/providers-and-models/ProvidersTab.tsx
apps/web/src/lib/ai-gateway/auto-model/index.ts
apps/web/src/lib/ai-gateway/auto-model/resolution.test.ts
apps/web/src/lib/ai-gateway/auto-model/resolution.ts
apps/web/src/lib/ai-gateway/llm-proxy-helpers.ts
apps/web/src/lib/ai-gateway/providers/openrouter/index.ts
apps/web/src/lib/organizations/organization-auto-model-shared.ts
apps/web/src/lib/organizations/organization-auto-model.ts - 1 issue
apps/web/src/lib/organizations/organization-base-types.ts
apps/web/src/lib/organizations/organization-models.ts
apps/web/src/lib/organizations/organization-modes.test.ts
apps/web/src/lib/organizations/organization-modes.ts
apps/web/src/lib/organizations/organization-seats.ts
apps/web/src/lib/organizations/organization-types.ts
apps/web/src/lib/organizations/organizations.ts
apps/web/src/lib/proxy-error-types.ts
apps/web/src/routers/organizations/organization-modes-router.test.ts
apps/web/src/routers/organizations/organization-modes-router.ts
apps/web/src/routers/organizations/organization-settings-router.test.ts
apps/web/src/routers/organizations/organization-settings-router.ts
packages/db/src/schema-types.test.ts
packages/db/src/schema-types.ts

Fix these issues in Kilo Cloud

_{Reviewed by gpt-5.4-20260305 · Input: 68K · Output: 4.4K · Cached: 162.3K}

_{Review guidance: REVIEW.md from base branch main}

pandemicsyn · 2026-06-17T12:22:04Z

Fixed review feedback in b3107a9: BYOK ownership validation now accepts and uses the active transaction client inside Organization Auto settings/mode mutations.

…ned-auto # Conflicts: # apps/web/src/app/api/openrouter/[...path]/route.ts

…ned-auto # Conflicts: # apps/web/src/lib/ai-gateway/providers/openrouter/index.ts

pandemicsyn · 2026-06-18T18:09:07Z

(will fix up the merge conflicts as they pop up - but not actively hacking on this beyond that)

…ned-auto

chrarnoldus · 2026-06-19T09:03:05Z

There are downsides of not telling the client which model is used:

Context window and multi-modal support may be off.
A suboptimal API format may be used.
If a modified prompt is available for the model family it will not be used.
Model variants will not be selectable (this is probably the easiest one to fix by making it part of the mode config).

Some of this can be mitigated by setting the opencode property if models happen to align, but I'm not sure there's an easy solution in general.

pandemicsyn added 3 commits June 16, 2026 16:55

feat(ai-gateway): add organization auto model routing

4d934f5

refactor(organizations): remove mode default model bridge

1162278

fix(organizations): harden organization auto routing

6e0261a

kilo-code-bot Bot reviewed Jun 17, 2026

View reviewed changes

Comment thread apps/web/src/lib/organizations/organization-auto-model.ts Outdated

fix(organizations): use transaction BYOK reads

b3107a9

pandemicsyn added 4 commits June 17, 2026 07:37

Merge remote-tracking branch 'origin/main' into florian/feat/org-defi…

ae97b04

…ned-auto # Conflicts: # apps/web/src/app/api/openrouter/[...path]/route.ts

Merge remote-tracking branch 'origin/main' into florian/feat/org-defi…

48d4c09

…ned-auto # Conflicts: # apps/web/src/lib/ai-gateway/providers/openrouter/index.ts

fix(ai-gateway): preserve auto model status

412123c

feat(organizations): unify default model behavior

29a354e

kilo-code-bot Bot reviewed Jun 17, 2026

View reviewed changes

Comment thread apps/web/src/routers/organizations/organization-settings-router.ts Outdated

pandemicsyn added 2 commits June 17, 2026 13:42

fix(organizations): improve default behavior chooser

cf3f457

fix(organizations): capture locked default for audit logs

f753b11

Merge remote-tracking branch 'origin/main' into florian/feat/org-defi…

115791b

…ned-auto

St0rmz1 reviewed Jun 18, 2026

View reviewed changes

Comment thread apps/web/src/lib/organizations/organization-auto-model.ts

St0rmz1 reviewed Jun 18, 2026

View reviewed changes

Comment thread apps/web/src/routers/organizations/organization-settings-router.ts Outdated

St0rmz1 reviewed Jun 18, 2026

View reviewed changes

Comment thread apps/web/src/routers/organizations/organization-modes-router.ts

St0rmz1 reviewed Jun 18, 2026

View reviewed changes

Comment thread apps/web/src/components/organizations/providers-and-models/DefaultModelDialog.tsx

fix(organizations): validate routed auto targets

9f9f610

pandemicsyn requested a review from St0rmz1 June 18, 2026 20:31

chrarnoldus reviewed Jun 19, 2026

View reviewed changes

Comment thread apps/web/src/lib/ai-gateway/auto-model/index.ts Outdated

chrarnoldus reviewed Jun 19, 2026

View reviewed changes

Comment thread apps/web/src/app/api/openrouter/[...path]/route.ts

chrarnoldus reviewed Jun 19, 2026

View reviewed changes

Comment thread apps/web/src/app/api/openrouter/[...path]/route.ts

refactor(ai-gateway): clarify organization auto routing

67a608c

Conversation

pandemicsyn commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Visual Changes

Reviewer Notes

Uh oh!

Uh oh!

kilo-code-bot Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Summary

Overview

WARNING

Previous review (commit 9f9f610)

Overview

Previous review (commit f753b11)

Executive Summary

Previous review (commit cf3f457)

Executive Summary

Previous review (commit 29a354e)

Executive Summary

Overview

SUGGESTION

Previous review (commit 412123c)

Executive Summary

Previous review (commit b3107a9)

Executive Summary

Previous review (commit 6e0261a)

Executive Summary

Overview

WARNING

Uh oh!

pandemicsyn commented Jun 17, 2026

Uh oh!

Uh oh!

pandemicsyn commented Jun 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chrarnoldus commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pandemicsyn commented Jun 17, 2026 •

edited

Loading

kilo-code-bot Bot commented Jun 17, 2026 •

edited

Loading

Previous review (commit `9f9f610`)

Previous review (commit `f753b11`)

Previous review (commit `cf3f457`)

Previous review (commit `29a354e`)

Previous review (commit `412123c`)

Previous review (commit `b3107a9`)

Previous review (commit `6e0261a`)

chrarnoldus commented Jun 19, 2026 •

edited

Loading